Information processing device and recording system

ABSTRACT

In an information processing device, a storage unit stores format information and an option value to be used in a recording process. A communication unit is configured to perform wireless communication with a service providing device and an image recording device configured to perform the recording process. A control unit executes: receiving layout definition data including layout information representing a layout of a content image icon; displaying a layout image in which the content image icon is disposed according to the layout information; receiving content data from the service providing device; displaying a preview image representing a result of the recording process when the user&#39;s touch operation is received on a location of the content image icon; and transmitting recording instruction when the user&#39;s operation for instructing the image recording device to perform the recording process is received.

CROSS REFERENCE TO RELATED APPLICATION

This application is continuation of U.S. Ser. No. 16/884,170 filed onMay 27, 2020, which is a continuation of U.S. Ser. No. 16/126,066 filedon Sep. 10, 2018, now U.S. Pat. No. 10,691,294 granted on Jun. 23, 2020,which is a continuation of U.S. Ser. No. 14/867,246, filed on Sep. 28,2015, now U.S. Pat. No. 10,073,588 granted on Sep. 11, 2018, which arebased upon and claim the benefit of priority of Japanese PatentApplication No. 2014-198117 filed on Sep. 29, 2014, the contents ofwhich are incorporated herein by reference in its entirety.

BACKGROUND

The present disclosure relates to a program capable of making an imagerecording device perform a recording process on content data acquiredfrom a service providing device.

In the related art, there is known a program capable of making an imagerecording device record images represented by data acquired from serviceproviding devices on sheets. For example, a program disclosed in arelated art can make an image recording device perform a recordingprocess of recording a character string representing a URL on a sheet ifthe character string representing the URL is included in a page acquiredfrom a WEB server, and a recording process of recoding the pagespecified by the corresponding URL on a sheet.

However, a page can include not only character string data but alsoother data of various data formats, such as image data, document data,and spreadsheet data. Also, in one page, plural pieces of data ofdifferent data formats may exist together.

SUMMARY

An aspect of the present disclosure was made in view of the abovedescribed circumstances, and an object of the aspect of the presentdisclosure is to provide a program capable of performing a recordingprocess with an appropriate option value according to the data format ofcontent data included in layout definition data.

The aspect of the present disclosure provides the followingconfigurations:

A non-transitory computer readable recording medium stores a computerreadable instruction which can be executed by a computer which includesa display unit, a storage unit, an operation unit for receiving user'stouch operations on a display screen of the display unit, and acommunication unit, wherein the computer is configured to performwireless communication through the communication unit and wherein theinstruction when executed by the computer causes the computer toexecute: a layout definition data receiving process of receiving layoutdefinition data from the service providing device through thecommunication unit, the layout definition data including layoutinformation representing a layout of a content image icon, wherein thecontent image icon is generated by and corresponds to content data,wherein the content data is stored in the service providing device; alayout displaying controlling process of controlling the display unit todisplay a layout image in which the content image icon is locatedaccording to the layout information; a content data receiving process ofreceiving content data from the service providing device through thecommunication unit; a first touch operation receiving process ofreceiving a user's touch operation on a location of the content imageicon by the operation unit; a preview displaying controlling process ofcontrolling the display unit to display a preview image, when the user'stouch operation on a location of the content image icon is receivedaccording to the first touch operation receiving process, wherein thepreview image represents an image recorded by a recording process to beperformed in an image recording device, wherein in the recording processthe image is recorded based on the content data according to an optionvalue corresponding to data format of the content data, wherein thecontent data corresponds to the touched content image icon, wherein theoption value is stored in the storage unit; an instruction operationreceiving process of receiving, by the operation unit, a user'soperation to perform the recording process; and a transmitting processof transmitting, to the image recording device through the communicationunit, recording instruction for instructing the image recording deviceto perform the recording process when the user's operation is receivedaccording to the instruction operation receiving process.

An information processing device comprises: a display unit; a storageunit; an operation unit that receives user's touch operations on adisplay screen of the display unit; a communication unit; and a controlunit. The storage unit stores format information representing a dataformat of content data which can be acquired from a service providingdevice, and an option value to be used in a recording process, inassociation with each other. The communication unit is configured toperform wireless communication with the service providing device and animage recording device configured to perform the recording process. Thecontrol unit is configured to execute: a layout definition datareceiving process of receiving layout definition data from the serviceproviding device through the communication unit, the layout definitiondata including layout information representing a layout of a contentimage icon, wherein the content image icon is generated by andcorresponds to content data, wherein the content data is stored in theservice providing device; a layout displaying controlling process ofcontrolling the display unit to display a layout image in which thecontent image icon is located according to the layout information; acontent data receiving process of receiving content data from theservice providing device through the communication unit; a first touchoperation receiving process of receiving a user's touch operation on alocation of the content image icon by the operation unit; a previewdisplaying controlling process of controlling the display unit todisplay a preview image when the user's touch operation on a location ofthe content image icon is received according to the first touchoperation receiving process, wherein the preview image represents animage recorded by a recording process to be performed in an imagerecording device, wherein in the recording process the image is recordedbased on the content data according to an option value corresponding todata format of the content data, wherein the content data corresponds tothe touched content image icon, wherein the option value is stored inthe storage unit; an instruction operation receiving process ofreceiving, by the operation unit, a user's operation; and an outputtingtransmitting process of transmitting, to the image recording devicethrough the communication unit, recording instruction for instructingthe image recording device to perform the recording process when theuser's operation is received according to the instruction operationreceiving process.

A recording system comprises: an image recording device including: arecording unit configured to perform a recording process of recordingimages on sheets; a second communication unit; and a second controlunit; and an information processing device including: a display unitincluding a display screen; a storage unit; an operation unit configuredto receive user's touch operations on the display screen; a firstcommunication unit configured to perform wireless communication with animage recording device and a service providing device; and a firstcontrol unit. The storage unit stores format information representing adata format of content data which can be acquired from the serviceproviding device, and condition information representing condition forperforming the recording process, in association with each other. Thefirst control unit performs the following processes: a layout definitiondata receiving process of receiving layout definition data from theservice providing device through the communication unit, the layoutdefinition data including layout information representing a layout of acontent image icon, wherein the content image icon is generated by andcorresponds to content data, wherein the content data is stored in theservice providing device; a layout displaying controlling process ofcontrolling the display unit to display a layout image in which thecontent image icon is located according to the layout information; acontent data receiving process of receiving content data from theservice providing device through the communication unit; a first touchoperation receiving process of receiving a user's touch operation on alocation of the content image icon by the operation unit; a previewdisplaying controlling process of controlling the display unit todisplay a preview image representing a result of the recording processwhen the user's touch operation on a location of the content image iconis received according to the first touch operation receiving process,wherein the preview image represents an image recorded by a recordingprocess to be performed in an image recording device, wherein in therecording process the image is recorded based on the content dataaccording to an option value corresponding to data format of the contentdata, wherein the content data corresponds to the touched content imageicon, wherein the option value is stored in the storage unit; aninstruction operation receiving process of receiving, by the operationunit, a user's operation to perform the recording process; and atransmitting process of transmitting, to the image recording devicethrough the communication unit, recording instruction for instructingthe image recording device to perform the recording process when theuser's operation is received according to the instruction operationreceiving process. The second control unit is configured execute: arecording instruction information receiving process of receiving therecording instruction from the information processing device through thesecond communication unit; and a recording controlling process ofcontrolling the recording unit to perform the recording processrepresented by the corresponding recording instruction when therecording instruction is received according to the recording instructioninformation receiving process.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of a recording system 100 according to anembodiment.

FIG. 2 shows examples of format information and option values stored ina data storage area 62B.

FIG. 3A shows an example of a layout definition data, and FIG. 3B showsan example of a theme information.

FIG. 4 is a flow chart of a note printing process of a first embodiment.

FIG. 5 is a flow chart of a note list displaying process.

FIG. 6 is a flow chart of a note displaying process.

FIGS. 7A and 7B show a service selection screen and a layout informationinput screen as display examples of a display unit 53, respectively.

FIGS. 8A and 8B show an example including a first thumbnail image 111and an example including a second thumbnail image 113 as displayexamples of a note selection screen of the display unit 53,respectively.

FIGS. 9A and 9B show an example including a format image 117 and anexample including a thumbnail image 118 as display examples of a notedisplay screen of the display unit 53, respectively.

FIGS. 10A and 10B show an example including a preview image 121 of acontent data having an image format and an example including a previewimage 122 having a format different from the image format, as displayexamples of a preview screen of the display unit 53, respectively.

FIGS. 11A and 11B shows a note preview screen and a text preview screenas display examples of the display unit 53, respectively.

FIG. 12 is a view for explaining a sharing function which is implementedby a common API.

FIG. 13 is a flow chart illustrating a note printing process accordingto a second embodiment.

FIG. 14 is a flow chart of a token acquiring process.

FIG. 15 is another example of the layout definition data.

FIGS. 16A and 16B show a preview screen including a layout image 125 anda preview screen including a layout image 126 as display examples of thedisplay unit 53, respectively.

FIG. 17 is a flow chart of a note printing process according to amodification of the second embodiment.

DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

Hereinafter, illustrative embodiments according to the presentdisclosure will be described with reference to appropriate drawings.Also, it goes without saying that the embodiments to be described beloware just examples of the present disclosure and the embodiments of thepresent disclosure can be appropriately modified without changing thescope of the present invention.

FIG. 1 is a schematic diagram of a recording system 100 according to anembodiment. The recording system 100 shown in FIG. 1 is composed of amulti-function device 10, a portable terminal 50, and service providingdevices 70 and 80. Meanwhile, a service providing device 90 will bedescribed below with respect to a second embodiment. The multi-functiondevice 10, the portable terminal 50, and the service providing devices70 and 80 can perform communication with one another through acommunication network 102. Specific examples of the communicationnetwork 102 are not particularly limited, and may be, for example, theInternet, a mobile communication network, a wired LAN (an abbreviationfor local area network), a wireless LAN, or a combination of them.

Multi-Function Device 10

As shown in FIG. 1 , the multi-function device 10 manly includes aprinter unit 11, a scanner unit 12, a display unit 23, an operation unit24, a communication unit 25, a CPU 31, a storage unit 32, and acommunication bus 33. These individual components constituting themulti-function device 10 are connected to one another through thecommunication bus 33. The multi-function device 10 is an example of animage recording device.

Printer Unit 11 and Scanner Unit 12

The printer unit 11 performs a recording process of recording imagesrepresented by image data on recording sheets which are examples ofsheets. The recording mode of the printer unit 11 is not particularlylimited, and can be a publicly known mode such as an inkjet mode or anelectrophotographic mode. The scanner unit 12 performs a scanningprocess of reading images recorded on recording sheets, therebygenerating image data. The multi-function device 10 may further haveother functions, such as a FAX function of performing transmission andreception of facsimiles or a copying function of reading images recordedon recording sheets and recording the read images on other recordingsheets.

Display Unit 23

The display unit 23 has a display screen which displays a variety ofinformation. The specific configuration of the display unit 23 is notparticularly limited, and may use, for example, an LCD (an abbreviationfor liquid crystal display), an OLED (an abbreviation for organicelectro-luminescence display), or the like.

Operation Unit 24

The operation unit 24 receives user's operations for selecting objectsdisplayed on the display screen of the display unit 23. Specifically,the operation unit 24 has, for example, push buttons, and outputsvarious operation signals associated with pushed push buttons, to theCPU 31. Also, the operation unit 24 may have a membranous touch sensorsuperimposed on the display screen of the display unit 23. That is, thedisplay unit 23 may be configured as a touch panel display. As the touchsensor, a publicly known type such as an electrostatic capacitance typeor a resistive film type can be used.

The term “object” means an image which the user can select by operatingthe operation unit 24. As an example, the objects may be characterstrings displayed on the display unit 23. In this case, if the userpushes direction keys of the operation unit 24, one of the objects maybe highlighted, and then if the user pushes a determination button ofthe operation unit 24, the highlighted object may be selected. Asanother example, in a case where the operation unit 24 is a touch panel,the objects may be icons, buttons, links, and the like displayed on thedisplay unit 23, and an object displayed at a touched position may beselected.

Communication Unit 25

The communication unit 25 is an interface for performing communicationwith external devices through the communication network 102. That is,the multi-function device 10 outputs a variety of information to theportable terminal 50 or the service providing device 70 or 80 throughthe communication unit 25, and receives a variety of data or a varietyof information from the portable terminal 50 or the service providingdevice 70 or 80 through the communication unit 25. A specificcommunication procedure of the communication unit 25 is not particularlylimited, and can use, for example, Wi-Fi (a trademark of the Wi-FiAlliance).

CPU 31

The CPU (an abbreviation for central processing unit) 31 controls theoperation of the whole multi-function device 10. On the basis of avariety of information which are output from the operation unit 24, avariety of information acquired from external devices through thecommunication unit 25, and the like, the CPU 31 acquires variousprograms (to be described below) from the storage unit 32, and executesthose programs. That is, the CPU 31 and the storage unit 32 constitutean example of a control unit.

Storage Unit 32

The storage unit 32 has a program storage area 32A and a data storagearea 32B. In the program storage area 32A, an OS (an abbreviation foroperating system) 34 and a control program 35 are stored. Also, thecontrol program 35 may be a single program, or may be an aggregate of aplurality of programs. In the data storage area 32B, data or informationnecessary for executing the control program 35 are stored. The datastorage area 32B stores format information and option values inassociation with each other as shown in FIG. 2 .

The format information is information representing the data format ofcontent data, text data, and layout definition data. In the example ofFIG. 2 , the format information represent “jpg” which is an example ofthe data format of image data, “pdf” which is an example of a dataformat different from the image format, “txt” which is an example of thedata format of text data, and “xml” which is an example of the dataformat of layout definition data. However, specific examples of theformat information are not limited thereto. Hereinafter, the data formatof image data will also be referred to as the image format.

The option values are values representing execution conditions underwhich the multi-function device 10 should perform a recording process ondata having a corresponding data format. In the example of FIG. 2 ,option values are “SHEET SIZE” option value (such as a letter size orA4), “SHEET TYPE” option value (such as glossy paper or plain paper),“COLOR” option value (such as full-color, 256-color, or monochrome), andso on. However, specific examples of the option values are not limitedthereto, and may include a sheet direction, a resolution, amagnification, the number of copies, and so on.

In this specification, the term “data” and the term “information” arethe same as each other in that they each represent a bit or a bit stringwhich can be handled by a computer. However, when the computer handles“data,” it is unnecessary for the computer to recognize the meanings ofindividual bits. In contrast, when the computer handles “information,”an operation of the computer is diverged according to the meanings ofindividual bits. Also, a term “instruction” represents a control signalfor prompting a device which is a transmission destination to perform asubsequent operation. An instruction may include “information” or mayhave characteristics as “information”.

Further, even if “data” or the “information” is changed in format (forexample, a text format, a binary format, a flag format, and so on) foreach computer, as long as it can be recognized that the meaning of thechanged data or information is identical to the meaning of the unchangeddata or information, the changed data or information are handled as dataor information identical to the unchanged data or information. Forexample, information indicating “two” may be saved in one computer as anASCII code “0x32” which is text format information, and may be saved inanother computer as a binary number “10” which is binary formatinformation.

However, the term “data” and the term “information” are not strictlydistinguished, and are permitted to be exceptionally handled. Forexample, data may be temporarily handled as information, or informationmay be temporarily handled as data. Further, while one is handled asdata in one device, the one may be handled as information in anotherdevice. Furthermore, information may be taken out from among data, ordata may be taken out from among information.

The storage unit 32 may be configured, for example, by a RAM (anabbreviation for random access memory), a ROM (an abbreviation for readonly memory), an EEPROM (an abbreviation for electrically erasableprogrammable read-only memory), an HDD (an abbreviation for hard diskdrive), a buffer which is included in the CPU 31, or a combination ofthem.

The storage unit 32 may be a computer-readable storage medium. Thecomputer-readable storage medium is a non-transitory medium. Examples ofthe non-transitory medium include not only the above described examplesbut also other recording media such as a CD-ROM and a DVD-ROM. Also, thenon-transitory medium is also a tangible medium. Meanwhile, an electricsignal which carries a program which is downloaded from a server or thelike on the Internet is a computer-readable signal medium which is onetype of computer-readable medium, but is not included in non-transitorycomputer-readable storage media.

The programs stored in the program storage area 32A are executed by theCPU 31. However, in this specification, sometimes, the operation of eachprogram will be described without mentioning the CPU 31. That is, in thefollowing description, a description meaning that a program “A” performsa process “A” may mean that the CPU 31 performs the process “A”described in the program “A”. This is common to the portable terminal 50to be described below.

The OS 34 is a basic program which provides an API (an abbreviation forapplication programming interface) for controlling the printer unit 11,the scanner unit 12, the display unit 23, the operation unit 24, thecommunication unit 25, and so on which are hardware constituting themulti-function device 10. That is, each of the above described programscontrols a corresponding piece of hardware by calling the API which isprovided by the OS 34. However, in this specification, the operation ofeach program will be described without mentioning the OS 34. That is, inthe following description, a description meaning that a program “B”controls hardware “C” may means that the program “B” controls thehardware “C” through the API of the OS 34. This is common to theportable terminal 50 to be described below.

Portable Terminal 50

As shown in FIG. 1 , the portable terminal 50 mainly includes a displayunit 53, an operation unit 54, a communication unit 55, a CPU 61, astorage unit 62, and a communication bus 63. The display unit 53, theoperation unit 54, the communication unit 55, the CPU 61, the storageunit 62, and the communication bus 63 which are included in the portableterminal 50 are identical to the display unit 23, the operation unit 24,the communication unit 25, the CPU 31, the storage unit 32, and thecommunication bus 33 which are included in the multi-function device 10,and thus repeated descriptions thereof will not be made. The CPU 61 andthe storage unit 62 is an example of a control unit.

The operation unit 54 of the portable terminal 50 includes a membranoustouch sensor superimposed on the display screen of the display unit 53.Further, the operation unit 54 receives a user's touch operation on thedisplay screen of the display unit 53. Furthermore, the operation unit54 outputs location information representing the location on the displayscreen touched by a user. Location information can be expressed ascoordinates (x, y) on an x-y plane where a direction toward the right isthe positive side of the x axis and a downward direction is the positiveside of the y axis. The operation unit 54 is an example of a receivingunit which receives user's operations.

In this specification, the term “touch” includes every operation ofbringing input media into contact with the display screen. That is, atap operation of detaching a touched input medium from the displayscreen in a predetermined time, a long touch operation of stopping atouched input medium on the display screen, a slide operation of slidinga touched input medium on the display screen, a flick operation ofsliding a touched input medium with acceleration equal to or higher thana threshold value, a pinch-in operation of sliding two input mediatouched at different locations on the display screen toward each other,a pinch-out operation of sliding two input media touched at differentlocations on the display screen away from each other, and the like aretouch examples. Also, the input media may be user's fingers, touch pens,etc.

The communication unit 55 performs communication with the multi-functiondevice 10 or the service providing device 70 or 80 through a mobilecommunication network or a wireless LAN. In a program storage area 62Aof the storage unit 62, an OS 64 and a control program 65 are stored.Specific examples of the portable terminal 50 are not limited to a smartphone, a mobile phone, a tablet terminal, and the like, and may be a PC(an abbreviation for personal computer) and so on. The portable terminal50 is an example of an information processing device.

Service Providing Devices 70 and 80

The service providing devices 70 and 80 are devices for providingservices to be used in the portable terminal 50. The service providingdevices 70 and 80 exist on the Internet, and provide various services inresponse to requests from the portable terminal 50. The serviceproviding device 70 is a device for providing a service, for example,“Evernote (a trademark of Evernote Corporation)”. The service providingdevice 80 is a device for providing a storage service of storing data infile units, for example, “Dropbox (a trademark of DropBox, Inc.)” or“Google Drive (Google is a trademark of Google, Inc.)”, or a general WEBserver. The service providing device 70 is an example of a first serviceproviding device or a specific service providing device, and the serviceproviding device 80 is an example of a second service providing device.

The service providing device 70 stores layout definition data and themeinformation in association with each other as shown in FIGS. 3A and 3B.A layout definition data and a theme information corresponding to eachother are uniquely identified, for example, by layout dataidentification information (hereinafter, referred to as a note ID). Thenumber of layout definition data which can be stored in the serviceproviding device 70 is not limited to one, and may be two or more. Also,the service providing device 70 stores first thumbnail image datacorresponding to layout definition data, and thumbnail image datacorresponding to content data having the image format.

Each layout definition data is a data defining the layout of a layoutimage. Each layout definition data includes layout informationrepresenting the layout of content images and texts. Content images areimages representing content data. Texts are character stringsrepresented by text data included in layout definition data. That is,layout images are generated by disposing content images and textsaccording to layout information. Theme information are informationrepresenting the themes of layout images which are defined bycorresponding layout definition data. Layout definition data or layoutimages correspond to so-called notes, and themes correspond to so-calledtitles.

<TEXT> tag, <FILE> tag, and an <IMAGE> tag shown in FIG. 3A are examplesof layout information. The <TEXT> tag is layout information representingtext layouts. The <FILE> tags and the <IMAGE> tag are layout informationrepresenting a content image layout representing content datarepresented by a “src”. The <IMAGE> tag corresponds to content datahaving the image format, and the <FILE> tag corresponds to content datahaving a format other than the image format. A file name at the end of aURL (an abbreviation for uniform resource locator) represented by a“src” is an example of an identification information for identifying acontent data.

A language for describing layout definition data is not particularlylimited, and layout definition data can be described, for example, inHTML (an abbreviation for HyperText Markup Language), XHTML (anabbreviation for Extensible HyperText Markup Language), or XML (anabbreviation for Extensible Markup Language). Also, theme informationmay be included in layout definition data, or may be storedindependently from layout definition data.

Operation of Recording System 100

With reference to FIGS. 4 to 11B, a note printing process according to afirst embodiment will be described. The note printing process is aprocess in which the portable terminal 50 having acquired a layoutdefinition data from the service providing device 70 controls themulti-function device 10 such that the multi-function device performs arecording process of recording content, layout images, or textsrepresented by content data on sheets.

First, in STEP S11, the control program 65 of the portable terminal 50controls the display unit 53 such that the display unit displays aservice selection screen. FIG. 7A shows an example of the serviceselection screen. The service selection screen shown in FIG. 7A includesa “FIRST SERVICE” icon and a “SECOND SERVICE” icon. A first service is aservice which the service providing device 70 provides, and a secondservice is a service which the service providing device 80 provides.However, it goes without saying that the number of service icons whichcan be included in the service selection screen is not limited to two.Thereafter, in STEP S12, the control program 65 stands by in a statewhere it can receive a user's operation for selecting one of the serviceicons included in the service selection screen through the operationunit 54. The processes of STEPS S11 and S12 are an example of a serviceselecting process.

If the operation unit 54 receives a user's tap operation on the locationof the “FIRST SERVICE” icon (“YES” in STEP S12), in STEP S13, thecontrol program 65 performs a token acquiring process. The tokenacquiring process is a process of acquiring a token representing thatthere is the authority to acquire data stored in the service providingdevice 70. The token is an example of authentication information.Details of the token acquiring process will be described with respect toa second embodiment. In the first embodiment, on the assumption that thetoken has been acquired, the following description will be made. STEPS13 is an example of an acquiring process.

Subsequently, in STEP S14, the control program 65 acquires a note listfrom the service providing device 70. For example, in STEP S14, thecontrol program 65 receives note IDs, theme information, and firstthumbnail image data corresponding to layout definition data and storedin the service providing device 70, from the service providing device 70through the communication unit 55. The layout definition data, the themeinformation, and the first thumbnail image data may be stored in theservice providing device 70 in association with one another.

The note IDs are information for identifying the layout definition data.The theme information are information representing the titles(hereinafter, also referred to as themes) of notes which are defined bythe layout definition data. The theme information may be text datarepresenting the titles of the notes. The first thumbnail image data isdata representing first thumbnail images which are the thumbnails of thenotes. Each first thumbnail image may be, for example, an image obtainedby reducing the size of a representative image which is one of imageswhich are included in a corresponding note. Alternatively, eachthumbnail image may be an image obtained by decreasing the resolution ofan original image.

More specifically, in STEP S14, the control program 65 transmits listrequest information including the token acquired in STEP S13, to theservice providing device 70 through the communication unit 55. Then, thecontrol program 65 receives theme information and first thumbnail imagedata as a response to the list request information. The process of STEPS14 is an example of a layout data list acquiring process.

Subsequently, in STEP S15, the control program 65 performs a note listdisplaying process. The note list displaying process is a process ofdisplaying a list of layout identification images corresponding to thelayout definition data on the display unit 53, thereby enabling the userto select one of the layout definition data stored in the serviceproviding device 70. With reference to FIG. 5 , the note list displayingprocess will be described in detail.

First, in STEP S31, the control program 65 determines whether a secondthumbnail image data corresponding to one of the theme informationacquired in STEP S14 is stored in the data storage area 62B. If acorresponding second thumbnail image data is not stored in the datastorage area 62B (“No” in STEP S31), in STEP S32, the control program 65uses a first thumbnail image as a layout identification imagecorresponding to the corresponding theme information.

Meanwhile, if a corresponding second thumbnail image data is stored inthe data storage area 62B (“Yes” in STEP S31), in STEP S33, the controlprogram 65 uses the corresponding second thumbnail image data as alayout identification image corresponding to the corresponding themeinformation. Second thumbnail images are images which are represented bysecond thumbnail image data which are generated in STEP S19 to bedescribed below. Each second thumbnail image may be, for example, animage which is obtained by reducing the size, resolution, or the like ofa layout image representing all corresponding notes.

Until layout identification images corresponding to all themeinformation are determined (“No” in STEP S34), the control program 65repeatedly performs the processes of STEPS S31 to S33). If all layoutidentification images are determined (“YES” in STEP S34), in STEP S35,the control program 65 controls the display unit 53 such that thedisplay unit displays a note selection screen. FIGS. 8A and 8B showexamples of the note selection screen. In the note selection screensshown in FIGS. 8A and 8B, lists of themes represented by themeinformation and layout identification images 111 to 113 corresponding tothe theme information are displayed.

The layout identification images 111 and 112 shown in FIG. 8A areexamples of first thumbnail images. The first thumbnail images may berepresentative images of notes. In other words, the first thumbnailimages may be parts of the notes. Meanwhile, the layout identificationimage 113 shown in FIG. 8B is an example of second thumbnail images.Each second thumbnail image may be an image representing the whole of anote. However, a first thumbnail image and a second thumbnail imagecorresponding to a common theme information may be identical.

Returning to FIG. 4 , in STEP S16, the control program 65 stands by in astate where it can receive a user's operation for selecting one of thelayout identification images 111 to 113 included in the note selectionscreen through the operation unit 54. The processes of STEPS S15 and S16are an example of a layout data selecting process. Also, in the noteselection screen, in association with the layout identification images111 to 113, in place of the theme information, other information such astexts included in layout definition data may be displayed. Also, thefirst thumbnail image data and the second thumbnail image data may beassociated with information other than the theme information, forexample, layout definition data or note IDs corresponding to the themeinformation. The theme information, the layout definition data, and thenote IDs are associated with one another on a one to one basis.Therefore, it can be said that something corresponding to themeinformation indirectly corresponds to a layout definition data or a noteID.

If the operation unit 54 receives a user's tap operation on the locationof one of the layout identification images 111 to 113 (“Yes” in STEPS16), in STEP S17, the control program 65 receives a layout definitiondata corresponding to the touched layout identification image, from theservice providing device 70 through the communication unit 55. Morespecifically, the control program 65 transmits note request informationincluding the token and a note ID, to the service providing device 70through the communication unit 55. Then, the control program 65 mayreceive a layout definition data specified by the corresponding note ID,as a response to the note request information.

Also, in STEP S17, the control program 65 receives a thumbnail imagedata corresponding to a content data having the image format defined bythe layout definition data, from the service providing device 70 throughthe communication unit 55. For example, the control program 65 canreceive the thumbnail image data together with the content data byincluding an option for requesting transmission of the thumbnail imagedata in an HTTP request using the URL of the content data. The processof STEP S17 is an example of a layout data acquiring process.

Subsequently, in STEP S18, the control program 65 performs a notedisplaying process. The note displaying process is a process ofcontrolling the display unit 53 such that the display unit displays anote display screen including a layout image, by analyzing the layoutdefinition data acquired in STEP S17. More specifically, the notedisplaying process is a process of controlling the display unit 53 suchthat the display unit displays a layout image, by disposing textsrepresented by text data and content images represented by content dataaccording to layout information. A procedure of controlling the displayunit 53 such that the display unit displays a note selection screenshown in FIG. 9A or 9B according to the layout definition data of FIG.3A will be described with reference to FIG. 6 .

First, if the layout definition data represents text data (“Yes” in STEPS41), in STEP S42, the control program 65 disposes texts represented bythe corresponding text data, on a layout image, according to the layoutinformation. As an example, the control program 65 disposes “2014/9/9Sunny” represented by a <TEXT> tag of the first row of FIG. 3A, on theright side of a layout image 119 in a left-right direction as shown inFIG. 9A. As another example, the control program 65 disposes “My familymade a lunch.” and “The lunch was delicious.” represented by <TEXT> tagsof the second and sixth rows of FIG. 3A, on the left side of the layoutimage 119 in the left-right direction.

If the layout definition data represents content data having the imageformat (“Yes” in STEP S43), in STEP S44, the control program 65 disposesa thumbnail image represented by the thumbnail image data acquired inSTEP S17, on the layout image according to the layout information. Forexample, the control program 65 disposes a thumbnail image 115corresponding to the corresponding content data, as a content imagecorresponding to “pic.jpg” shown in the <IMAGE> tag of the fifth row ofFIG. 3A, at the center of the layout image 119 in the left-rightdirection, as shown in FIG. 9A.

If the layout definition data represents content data having a dataformat which is different from the image format and on which themulti-function device 10 cannot perform a recording process (“Yes” inSTEP S45), in STEP S46, the control program 65 disposes non-printingimages stored in the data storage area 62B, and the identificationinformation of the corresponding content data, on the layout image,according to the layout information. For example, the control program 65disposes a non-printing image 116 and a character string “COOKINGVIDEO.mpeg”, as a content image corresponding to “COOKING VIDEO.mpeg”shown in a <FILE> tag of the fourth row of FIG. 3A, on the left side ofthe layout image 119 in the left-right direction, as shown in FIG. 9A.The non-printing image 116 is an example of a non-compliant image onwhich a recording process cannot be performed. A video format and asound format are examples of a data format on which a recording processcannot be performed.

If the layout definition data represents content data having a dataformat which is different from the image format and on which themulti-function device 10 can perform a recording process (“No” in STEPS45), in STEP S47, the control program 65 determines whether anythumbnail image data corresponding to the corresponding content data isstored in the data storage area 62B. Also, a thumbnail image datacorresponding to content data is generated in STEP S23 to be describedbelow.

If any corresponding thumbnail image data is not stored in the datastorage area 62B (“No” in STEP S47), in STEP S49, the control program 65disposes a format image corresponding to the data format of thecorresponding content data, and the identification information of thecorresponding content data on the layout image according to the layoutinformation. For example, the control program 65 disposes a format image117 and a character string “RECIPE.pdf” as a content image correspondingto “RECIPE.pdf” shown in the <FILE> tag of the third row of FIG. 3A, onthe left side of the layout image 119 in the left-right direction, asshown in FIG. 9A.

Meanwhile, if a corresponding thumbnail image data is stored in the datastorage area 62B (“Yes” in STEP S47), in STEP S48, the control program65 disposes a thumbnail image represented by the corresponding thumbnailimage data, and the identification information of the correspondingcontent data, on the layout image according to the layout information.For example, the control program 65 disposes a thumbnail image 118 and acharacter string “RECIPE.pdf” as a content image corresponding to“RECIPE.pdf” shown in the <FILE> tag of the third row of FIG. 3A, on theleft side of a layout image 120 in a left-right direction, as shown inFIG. 9B.

Until analyzing of the whole layout information included in the layoutdefinition data finishes (“No” in STEP S50), the control program 65repeatedly performs the processes of STEPS S41 to S49. Further, thecontrol program 65 disposes a theme represented by a theme informationcorresponding to the layout definition data, at a predetermined locationon the layout image 119 or 120. The theme “LUNCH” of the firstembodiment is located at the upper left corner of the layout image 119or 120. As a result, the note display screen shown in FIG. 9A or 9B isdisplayed on the display unit 53. Also, the control program 65 maygenerate a layout image data representing the whole of the layout image119 or 120 according to the layout definition data, and control thedisplay unit 53 such that the control unit displays the layout image 119or 120 represented by the layout image data. The layout image data mayhave, for example, a JPEG format.

The note display screen shown in FIG. 9A includes the layout image 119,a “LAYOUT PRINT” icon, and a “TEXT PRINT” icon. The note display screenshown in FIG. 9B is different from the note display screen shown in FIG.9A in that it includes the layout image 120, instead of the layout image119. The layout image 120 is different from the layout image 119 in thatit includes the thumbnail image 118, instead of the format image 117.Subsequently, in STEP S20, the control program 65 stands by in a statewhere it can receive a user's operation on the note display screenthrough the operation unit 54. The processes of STEPS S18 and S20 areexamples of a layout displaying process.

In STEP S19, the control program 65 generates a second thumbnail imagedata corresponding to the layout image 119 or 120 displayed on thedisplay unit 53 in STEP S18. For example, in STEP S19, specifically, thecontrol program 65 generates a second thumbnail image data representingthe layout image 119 or 120 using the layout definition data, and storesthe corresponding second thumbnail image data in the data storage area62B in association with the note ID, the theme information, and thelike.

As an example, the control program 65 may generate a thumbnail imagedata by reducing the size, resolution or the like of the layout imagedata generated for displaying the note display screen, and store thecorresponding thumbnail image data in the data storage area 62B. Asanother example, the control program 65 may store the layout image dataas a thumbnail image data, in the data storage area 62B. However, in acase where a second thumbnail image data corresponding to the themeinformation is stored in the data storage area 62B, STEP S19 may beomitted. The process of STEP S19 is an example of a thumbnail generatingprocess.

If the operation unit 54 receives a user's tap operation on the locationof a content image included in the note display screen (“CONTENT IMAGE”in FIG. 20 ), in STEP S21, the control program 65 receives a contentdata corresponding to the content image, from the service providingdevice 70 through the communication unit 55. The process of STEP S21 isan example of a content data acquiring process.

For example, in STEP S21, the control program 65 transmits contentrequest information including the token and content data identificationinformation, to the service providing device 70 through thecommunication unit 55. Then, the control program 65 receives a contentdata identified by the corresponding identification information, as aresponse to the content request information. However, a content dataacquisition timing is not limited to STEP S21. For example, in STEP S17,every content data defined by the layout definition data may beacquired.

Subsequently, in STEP S22, the control program 65 controls the displayunit 53 such that the display unit displays a preview screen of thecontent data acquired in STEP S21. FIGS. 10A and 10B are examples of thepreview screen. The preview screen shown in FIG. 10A or 10B includes apreview image 121 or 122 representing a result which can be obtained byperforming a recording process on the content data according to anoption value corresponding to the content data, and a “PRINT” icon.Thereafter, in STEP S26, the control program 65 stands by in a statewhere it can receive a user's operation for selecting the “PRINT” iconthrough the operation unit 54. The process of STEP S22 is an example ofa preview displaying process.

For example, in STEP S22, the control program 65 reflects an optionvalue corresponding to the data format of the content data in the imagerepresented by the content data, thereby generating a preview imagedata, and includes the preview image 121 or 122 represented by thepreview image data, in the preview screen. Also, in STEP S27 (to bedescribed below), the control program 65 may transmit the preview imagedata generated for displaying the preview screen, to the multi-functiondevice 10 through the communication unit 55. The preview image data mayhave, for example, a JPEG format.

FIG. 10A is an example of a preview screen in a case where the operationunit 54 has received a user's tap operation on the location of thethumbnail image 115 shown in FIG. 9A or 9B. That is, the preview image121 represents a result which can be obtained by performing a recordingprocess on the content data “pic.jpg” acquired in STEP S21 according toan option value corresponding to its data format “jpg”. The previewimage 121 according to the present embodiment represents an image of aresult which can be obtained by recording an image represented by thecontent data “pic.jpg” in full color on a glossy paper sheet having aletter size.

FIG. 10B is an example of a preview image in a case where the operationunit 54 has received a user's tap operation on the location of theformat image 117 shown in FIG. 9A or the thumbnail image 118 shown inFIG. 9B. That is, the preview image 122 shows a result which can beobtained by performing a recording process on the content data“RECIPE.pdf” acquired in STEP S21 according to an option valuecorresponding to its data format “pdf”. The preview image 122 accordingto the present embodiment represents an image of a result which can beobtained by converting an image represented by the content data“RECIPE.pdf” into a 256-color image, and recording the 256-color imageon an A4 sheet.

Also, in a case where the content data is composed of a plurality ofpages, in the preview screen, a list of a first preview image 112Arepresenting the result of a recording process on a first page and asecond preview image 112B representing the result of a recording processon a second page. Also, in a case where page break informationrepresenting the location of the boundary between the first page and thesecond page is included in the content data, the control program 65 maygenerate the first preview image 112A and the second preview image 112Baccording to the page break information. Meanwhile, in a case where pagebreak information is not included in the content data, the controlprogram 65 may determine the location of the boundary between the pages,for example, on the basis of the number of rows which can be included ina corresponding sheet size or one page.

In STEP S23, the control program 65 generates a thumbnail image datacorresponding to the content data acquired in STEP S21, and stores thegenerated thumbnail image data in the data storage area 62B. A specificmethod of generating a thumbnail image data is not particularly limited.For example, in a case where the content data is composed of a pluralityof pages as shown in FIG. 10B, a thumbnail image data representing thecontent of the first page may be generated. Meanwhile, generation of athumbnail image data corresponding to a content data having the imageformat can be omitted. The process of STEP S23 is an example of thethumbnail generating process.

If the operation unit 54 receives a user's tap operation on the locationof the “PRINT” icon (“Yes” in STEP S26), in STEP S27, the controlprogram 65 transmits recording instruction information to themulti-function device 10 through the communication unit 55. Therecording instruction information is information for instructing themulti-function device 10 to perform the recording process for which theresult has been represented by the preview image 121 or 122. The processof STEP S27 is an example of an outputting process.

As an example, the recording instruction information includes print datafor controlling the multi-function device 10 such that themulti-function device performs a recording process. The print data isdata for performing a recording process on the content data acquired inSTEP S21 according to an option value corresponding to the data formatof the corresponding content data. In this case, the print data isgenerated by the control program 65. As another example, the recordinginstruction information includes the content data acquired in STEP S21,or a preview image data representing the preview image 121 or 122displayed in STEP S22, together with an option value corresponding tothe data format of the corresponding content data. In this case, theprint data is generated by the control program 35 of the multi-functiondevice 10.

If the operation unit 54 receives a user's tap operation on the locationof “LAYOUT PRINT” icon included in the note display screen (“LAYOUTPRINT” icon in STEP S20), in STEP S24, the control program 65 controlsthe display unit 53 such that the display unit displays a note previewscreen. FIG. 11A is an example of a note preview screen in a case wherethe layout image 119 shown in FIG. 9A has been selected. The previewscreen shown in FIG. 11A includes a preview image 123 and a “PRINT”icon.

The preview image 123 shows a result which can be obtained by performinga recording process on the layout image 119 according to an option valuecorresponding to the data format “xml” of the layout definition data.The option value corresponding to the data format of the layoutdefinition data is an example of layout image option value. However,among content images included in the layout image 119, a content imagecorresponding to content data having a data format on which a recordingprocess cannot be performed are not reflected in the preview image 123.That is, in the preview image 123, the non-printing image 116 andidentification information corresponding to the content data “COOKINGVIDEO.mpeg” are not included.

In other words, among the content images included in the layout image119, only a content image corresponding to content data having a dataformat on which a recording process can be performed are reflected inthe preview image 123. That is, in the preview image 123, textsrepresented by text data, the format image 117 and identificationinformation corresponding to the content data “RECIPE.pdf”, and thethumbnail image 115 and identification information corresponding to thecontent data “pic.jpg” are included. Also, in a preview screen which isdisplayed if the layout image 120 shown in FIG. 9B is selected, insteadof the format image 117, the thumbnail image 118 is included.

If the operation unit 54 receives a user's tap operation on the locationof the “TEXT PRINT” icon included in the note display screen (“TEXTPRINT” icon in STEP S20), in STEP S25, the control program 65 controlsthe display unit 53 such that the display unit displays a text previewscreen. FIG. 11B is an example of the text preview screen. The textpreview screen shown in FIG. 11B includes a preview image 124 and a“PRINT” icon.

The preview image 124 is an image representing a result which can beobtained by performing a recording process on texts extracted from thelayout definition data according to an option value corresponding to thedata format “txt” of text data. The option value corresponding to thedata format of the text data is an example of text option value. Also,the texts extracted from the layout definition data may be locatedaccording to the layout information or may be located always on the leftside.

Subsequently, if the operation unit 54 receives a user's tap operationon the location of the “PRINT” icon (“YES” in STEP S26), in STEP S27,the control program 65 transmits recording instruction information forinstructing the multi-function device to perform the recording processfor which the result has been represented by the preview image 123 or124.

Meanwhile, if the control program 35 of the multi-function device 10receives the recording instruction information from the portableterminal 50 through the communication unit 25, the control program 35controls the printer unit 11 such that the printer unit performs therecording process represented by the corresponding recording instructioninformation. Also, as described above, data which is included in therecording instruction information may be the content data, or may be thepreview image data representing the preview image, or may be the printdata. That is, as long as the multi-function device 10 can perform therecording process according to the recording instruction information,specific examples of the data which is included in the recordinginstruction information are not particularly limited.

Meanwhile, if the operation unit 54 receives a user's tap operation onthe location of the “SECOND SERVICE” icon (“No” in STEP S12), in STEPS28, the control program 65 transmits a request to the service providingdevice 80 by a method according to a corresponding service. For example,in a case where the service providing device 80 is a general WEB server,the control program 65 may transmit a general HTTP request to theservice providing device 80 through the communication unit 55.

Effects of First Embodiment

According to the first embodiment, according to an option valuecorresponding to the data format of a selected content data, a recordingprocess on the corresponding content data is performed. That is, withappropriate option value according to a data format, a recording processis performed. The control program 65 may receive a user's operation forchanging an option value stored in the data storage area 62B, throughthe operation unit 54. In this case, with the option value desired bythe user, a recording process is performed.

According to the first embodiment, in response to selection of a contentdata on the layout image 119 or 120 (that is, a touch on a contentimage), the preview image 121 or 122 of the corresponding content datais displayed on the display unit 53. Therefore, the user can easilyselect content to be a target of a recording process, in the layoutimage 119 or 120 laid out such that it can be seen easily, and can inputa recording instruction after checking the result of the recordingprocess in a preview screen.

According to the first embodiment, as a content image representing acontent data having the image format, a thumbnail image is used.Therefore, the user can check the content of content data having theimage format on layout images. According to the first embodiment,according to the propriety of performance of a recording process, aformat image or a non-printing image is used as a content imagerepresenting a content data of a format other than the image format.Therefore, the user can recognize the propriety of performance of arecording process at a glance. Further, according to the firstembodiment, a thumbnail image of a content data whose format is not theimage format is generated for displaying a preview image. Thereafter,when a layout image is displayed, instead of a format image, thethumbnail image is displayed. Therefore, even with respect to contentdata having formats other than the image format, the user can check thecontent of the corresponding content data on the layout images. As aresult, the user can more quickly select a desired content data.

A method of selecting a content image on a layout image is not limitedto a tap operation, and may be, for example, a long touch operation or adouble-tap operation. Check boxes may be provided for content images,respectively, such that a plurality of content data can be collectivelyselected by inputting check marks in corresponding check boxes. In thiscase, preview images of the selected content data are displayed as alist on a preview screen. Thereafter, if the “PRINT” icon is touched, arecording process on each content data is performed. In this way, it ispossible to collectively perform a recording process on a plurality ofcontent data included in a layout image. Therefore, it is possible toperform a recording process on a large number of pieces of content databy few operations

According to the first embodiment, it is possible to record not only asingle content data but also a layout image having content images laidout therein or texts included in a layout definition data, on sheets.Also, according to the first embodiment, content images representingcontent data on which a recording process cannot be performed are notreflected in a preview image, and are not recorded on recording sheets.However, it may be possible to enable the user to select whether toreflect each content data, representing a content data on which arecording process cannot be performed, in a preview image.

STEPS S23, S24, and S25 may be arbitrarily combined, and thecorresponding combination may be performed. As an example, if the“LAYOUT PRINT” icon included in the note display screen is selected,STEPS S23 and S24 may be preformed. In this case, a recording sheethaving a layout image recorded thereon, and recording sheets havingcontent data recorded individually thereon are output from the printerunit 11. As another example, if the “TEXT PRINT” icon included in thenote display screen is selected, STEPS S23 and S25 may be performed. Inthis case, recording sheets having only texts recorded thereon, andrecording sheets having content data recorded individually thereon areoutput from the printer unit 11.

According to the first embodiment, a thumbnail image of a note displayedon the note display screen is generated. Thereafter, in the note displayscreen which is displayed, the corresponding thumbnail image isdisplayed as a layout identification image. Therefore, the user canconfirm the content of each layout definition data. As a result, theuser can quickly select a desired layout definition data.

Second Embodiment

Now, with reference to FIGS. 12 to 16B, a note printing processaccording to a second element will be described. Also, components andprocesses identical to those of the first embodiment are denoted by thesame reference symbols, and will not be described. Differences of thesecond embodiment from the first embodiment will be mainly described. Acontrol program 65 which performs the note printing process according tothe second embodiment is different from that of the first embodiment inthat it uses a sharing function which is provided by the OS 64 and isactivated by another application. Also, the note printing processaccording to the second embodiment is different from that of the firstembodiment in that according to whether it is possible to acquire atoken, layout definition data acquisition sources are switched.

First, with reference to FIG. 12 , the sharing function which isimplemented by an API which is provided by the OS 64 of the portableterminal 50 will be described. The sharing function is a function ofcontrolling a sharing destination application such that the sharingdestination application processes data output from a sharing sourceapplication.

First, in STEP S1, a sharing source application 41 performs an API(hereinafter, referred to as the “sharing API”) of the OS 64 forimplementing the sharing function. By performing the sharing API, URI(an abbreviation for uniform resource identifier) information, typeinformation, and action information are obtained as input arguments. TheURI information is information representing the location of sharingdata. The type information is information representing the data formatof the shared data. The action information is information representingthe type of processing (such as browsing, editing, transmitting, orprinting) which is required to be performed in the sharing destinationapplication.

Meanwhile, with respect to applications 42, 43, 44 . . . to becandidates for the sharing destination application, data formats whichthey can process, and the types of processing which they can perform arestated in advance in the OS 64. The OS 64 specifies applications whichcan perform processing represented by the action information on thesharing data having the data format represented by the type information,from among the applications 42, 43, 44 . . . in STEP S2, and controlsthe display unit 53 in STEP S3 such that the display unit displays alist of the specified applications. Then, in STEP S4, the OS 64 standsby in a state where it can receive a user's operation for selecting oneof the applications displayed in the list through the operation unit 54.

If the operation unit 54 receives a user's operation for selecting oneof the applications displayed in the list in STEP S4, in STEP S5, the OS64 activates the corresponding application using the URI information,the type information, and the action information as input arguments. Onthe assumption that in the second embodiment, in STEP S5, the controlprogram 65 for performing a note printing process shown in FIG. 13 isactive, the following processes will be described. Also, the OS 64 maynotify the application 41 of information representing the applicationspecified in STEP S2, and control the application 41 such that theapplication 41 performs the processes of STEPS S3 to S5.

In STEP S51, the control program 65 activated by the sharing functionacquires the sharing data represented by a URI given as an inputargument. The URI represents, for example, a file path for a text filestored in the data storage area 62B. Further, this text file can includea URL representing the location of a layout definition data stored inthe service providing device 70, and a theme information correspondingto the corresponding layout definition data. Furthermore, the URLrepresenting the location of the layout definition data, for example,like “www.server1.com/shared/?id=001” includes a note ID “001” foridentifying the corresponding layout definition data on the serviceproviding device 70. The process of STEP S51 is an example of a locationinformation acquiring process. Also, “www.server1.com” represents theservice providing device 70. Meanwhile, “www.server2.com2” representsthe service providing device 80.

The service providing device 70 according to the second embodimentstores a first layout definition data shown in FIG. 3A and a secondlayout definition data shown in FIG. 15 as layout definition data whichare identified by the note ID “001”. The second layout definition datais a data obtained by performing processing on the corresponding firstlayout definition data. The second layout definition data according tothe second embodiment is obtained by adding an <IMAGE> tag, where“header.jpg” and “footer.jpg” has been set as an “src”, to the firstlayout definition data.

Further, in the second embodiment, in addition to the service providingdevices 70 and 80, the service providing device 90 for providing aservice identical to that of the service providing device 70 exists.However, the service providing devices 70 and 90 independently providethe services to users in different areas. Therefore, a token of theservice providing device 70 cannot be used to access to data stored inthe service providing device 90. Also, “www.server2.com2” is a URLrepresenting the service providing device 90.

If the URL acquired in STEP S51 represents the location of the serviceproviding device 70 or 90 (“Yes” in STEP S52), in STEP S53, the controlprogram 65 performs a token acquiring process. With reference to FIG. 14, details of the token acquiring process will be described. Also, if acharacter string representing a service providing device is included inthe URL, the control program 65 determines that the corresponding URLrepresents the location of the corresponding service providing device.Specifically, in a case where the URL includes “www.server1.com” or“www.server2.com2”, the control program 65 determines that thecorresponding URL represents the location of the service providingdevice 70 or 90.

First, in STEP S71, the control program 65 determines whether the atoken for accessing the service providing device represented by the URLacquired in STEP S51 is stored in the data storage area 62B. If acorresponding token is not stored in the data storage area 62B (“No” inSTEP S71), in STEP S72, the control program 65 determines whetheraccount information for acquiring a token is stored in the data storagearea 62B. The processes of STEPS S71 and S72 are examples of adetermining process.

If account information is not stored in the data storage area 62B (“No”in STEP S72, in STEP S73, the control program 65 controls the displayunit 53 such that the display unit displays an account information inputscreen. FIG. 7B is an example of the account information input screen.The account information input screen shown in FIG. 7B includes a message“INPUT A USER ID AND A PASSWORD.”, a text box for receiving a user IDinput, a text box for receiving a password input, and a “LOGIN” icon.Then, in STEP S74, the control program 65 stands by in a state where itcan receive a user's operation on the display unit 53 displaying theaccount information input screen through the operation unit 54. Theprocesses of STEPS S73 and S74 are an example of an account informationacquiring process.

If the operation unit 54 receives a user's tap operation on the locationof the “LOGIN” icon (“Yes” in STEP S74), in STEP S75, the controlprogram 65 acquires a token corresponding to the combination of a userID and a password input in the text boxes, from the service providingdevice 70 or 90. The user ID and the password are examples of accountinformation for acquiring a token. The process of STEP S75 is an exampleof an authentication information acquiring process. Also, a tokenacquisition source is not limited to the service providing device 70 or90, and may be any other device which determines the validity of accountinformation and issues a token corresponding to the service providingdevice 70 or 90. The token acquisition source will also be referred toas a token issuing device.

The OS 64 provides an API (hereinafter, referred to as the “token API”)for requesting a token of the service providing device 70 or 90. Thecontrol program 65 can use the token API to transmit account informationto the token issuing device and acquire a token as a response to thetransmission of the account information. Also, the token issuing deviceissues a token corresponding to an area where the portable terminal 50exists. That is, when STEP S75 is performed, in a case where theportable terminal 50 is in an area corresponding to the serviceproviding device 70, a token corresponding to the service providingdevice 70 is acquired; whereas in a case where the portable terminal 50is in an area corresponding to the service providing device 90, a tokencorresponding to the service providing device 90 is acquired.

For example, in STEP S75, the control program 65 transmitsauthentication request information including the user ID and thepassword to the token issuing device through the communication unit 55.Then, the control program 65 receives a token, as a response to theauthentication request information, from the token issuing devicethrough the communication unit 55. Also, in STEP S75, the controlprogram 65 stores the user ID and the password acquired from the userthrough the account information input screen, and the token acquiredfrom the token issuing device, in the data storage area 62B.

If a token is acquired from the token issuing device in STEP S75 (“Yes”in STEP S76), in STEP S77, the control program 65 determines whether thetoken is a token of the service providing device represented by the URLacquired in STEP S51. In other words, in STEP S77, the control program65 determines whether the service providing device 70 or 90 whoselocation is represented by the URL acquired in STEP S51 coincides withthe service providing device 70 or 90 which can be accessed to acquiredata by the token acquired in STEP S75.

If the acquired token is a toke of the service providing devicerepresented by the URL acquired in STEP S51 (“Yes” in STEP S77), in STEPS78, the control program 65 determines that token acquisition wassuccessful, and finishes the token acquiring process. Meanwhile, if theacquired token is not a token of the service providing devicerepresented by the URL acquired in STEP S51 (“No” in STEP S77), in STEPS79, the control program 65 determines that token acquisition failed andfinishes the token acquiring process. Also, if a token cannot beacquired in STEP S75 (“No” in STEP S76), in STEP S79, the controlprogram 65 determines that token acquisition failed and finishes thetoken acquiring process.

If account information is stored in the data storage area 62B (“Yes” inSTEP S72), the control program 65 skips the processes of STEPS S73 andS74, and performs the processes of STEP S75 and the subsequent stepsusing the stored account information. Further, if a token is stored inthe data storage area 62B (“Yes” in STEP S71), the control program 65skips the processes of STEPS S72 to S76, and performs the processes ofSTEP S77 and the subsequent steps with respect to the stored token.

Returning to FIG. 13 , if token acquisition is successful in STEP S53(“Yes” in STEP S54), in STEP S55, the control program 65 acquires afirst layout definition data. For example, the control program 65transmits note request information including the token and the note ID“001” included in the URL acquired in STEP S51, to the service providingdevice 70 through the communication unit 55. Then, the control program65 receives a first layout information identified by the note ID “001”,as a response to the note request information, from the serviceproviding device 70 through the communication unit 55. The process ofSTEP S55 is an example of a first acquiring process. The first layoutdefinition data is an example of first data.

Subsequently, in STEP S56, the control program 65 determines whether adescription of a theme information corresponding to the note ID “001” isincluded in a text file which was referred to in STEP S51. If adescription of a theme information is not included in the correspondingtext file (“No” in STEP S56), in STEP S57, the control program 65acquires a corresponding theme information from the service providingdevice 70. Meanwhile, if a description of a theme information isincluded in the corresponding text file (“Yes” in STEP S56), the controlprogram 65 skips the process of STEP S57.

For example, in STEP S57, the control program 65 transmits theme requestinformation including the note ID “001” to the service providing device70 through the communication unit 55. Then, the control program 65receives a theme information corresponding to the note ID “001”, as aresponse to the theme request information, from the service providingdevice 70 through the communication unit 55. The process of STEP S57 isan example of a theme information acquiring process.

Subsequently, in STEP S58, the control program 65 controls the displayunit 53 such that the display unit displays a preview screen shown inFIG. 16A. The preview screen shown in FIG. 16A includes the layout image125 and a “PRINT” icon. The layout image 125 is generated by analyzingthe first layout definition data acquired in STEP S55. Also, at theupper left corner of the layout image 125, a theme represented by thetheme information is located. The layout image 125 is an example of afirst layout image. The process of STEP S58 is an example of the previewdisplaying process.

If token acquisition fails in STEP S53 (“No” in STEP S54), in STEP S59,the control program 65 acquires a second layout definition data from alocation represented by the URL acquired in STEP S51. For example, inSTEP S59, the control program 65 can acquire the second layoutdefinition data by a general HTTP request. Also, a second layoutdefinition data acquisition source may be the service providing device70, or may be a service providing device different from the serviceproviding device 70. STEP S59 is an example of a second acquiringprocess. The second layout definition data is an example of second data.

Subsequently, in STEP S60, the control program 65 controls the displayunit 53 such that the display unit displays a preview screen shown inFIG. 16B. The preview screen shown in FIG. 16B includes the layout image126 and a “PRINT” icon. The layout image 126 is generated by analyzingthe second layout definition data acquired in STEP S59. The layout image126 is an example of a second layout image. The process of STEP S60 isanother example of the preview displaying process.

The layout image 126 is different from the layout image 125 in that itincludes a content image 127 corresponding to content data “Header.jpg”and a content image 128 corresponding to content data “footer.jpg”. As aresult, a part of the layout image 126 corresponding to a note issmaller than that of the layout image 125. Also, the layout image 126includes images corresponding to the thumbnail image 115, thenon-printing image 116, and the format image 117 included in the layoutimage 125. However, even if the operation unit 54 receives a user's tapoperation on any one of those images, a process of acquiring a contentdata corresponding to the corresponding image is not performed. Further,the layout image 126 does not include a theme.

Subsequently, in STEP S61, the control program 65 stands by in a statewhere it can receive a user's operation on the preview screen throughthe operation unit 54. If the operation unit 54 receives a user's tapoperation on the location of the “PRINT” icon (“Yes” in STEP S61), inSTEP S62, the control program 65 transmits recording instructioninformation to the multi-function device 10 through the communicationunit 55. The recording instruction information which is transmitted inSTEP S62 is information for instructing the multi-function device 10 toperform a recording process on the layout image 125 or 126. The processof STEP S62 is an example of the outputting process.

Meanwhile, if the URL acquired in STEP S51 represents the location ofthe service providing device 80 different from the service providingdevices 70 and 90 (“No” in STEP S52), in STEP S63, the control program65 transmits a request to the service providing device 80, using theURL, without performing the token acquiring process. Specifically, thecontrol program 65 may transmit a HTTP Get request using the URL. Then,the control program 65 performs processing on data acquired as aresponse to the corresponding request.

As an example, if the control program 65 acquires a layout definitiondata as a response to the HTTP Get request, it controls the display unit53 such that the display unit displays a layout image acquired byanalyzing the corresponding layout definition data. Also, for example,even in STEP S59, by the same method (for example, an HTTP Get request)as that in STEP S63, the control program 65 may acquire a second layoutdefinition data from the service providing device 90. That is, in STEPSS59 and S60 and STEP S63, the control program 65 may perform processingin the same procedure.

Meanwhile, if a data which is not a layout definition data is acquiredas a response to the HTTP Get request, the control program 65 performsprocessing according to the corresponding data. For example, in a casewhere the URL acquired in STEP S51 represents a content data of theservice providing device 80, in STEP S63, the control program 65 mayacquire the content data represented by the corresponding URL from theservice providing device 80 and control the display unit 53 such thatthe display unit displays a content image represented by thecorresponding content data.

Effects of Second Embodiment

According to the second embodiment, in a case where a token is stored inthe data storage area 62B, the first acquiring process (STEP S55) isperformed, whereby the layout image 125 which does not include thecontent images 127 and 128 can be recorded on a sheet. Meanwhile, in acase where a token is not stored in the data storage area 62B, thesecond acquiring process (STEP S59) is performed, whereby the layoutimage 126 including the content images 127 and 128 can be recorded on asheet. As a result, according to whether there is a token, it ispossible to record an appropriate layout image on a sheet.

The layout image 126 according to the second embodiment is obtained byincluding the content images 127 and 128 in the layout image 125.However, layout image processing is not limited to content imageaddition, and may include image processing including changing of theresolution or number of colors of a layout image and the like. Also,data which are acquired in the first acquiring process and the secondacquiring process are not limited to layout definition data. Forexample, in the second acquiring process, the control program 65 mayacquire a layout image data representing the layout image 126.

According to the token acquiring process related to the secondembodiment, in a case where a token is not stored in the data storagearea 62B, a token is acquired using account information, and in a casewhere account information is not stored in the data storage area 62B,account information is acquired from the user. Therefore, a possibilitythat a recording process on the layout image 125 which does not includethe content images 127 and 128 will be performed increases.

In the token acquiring process according to the second embodiment, atoken of a service providing device corresponding to an area where theportable terminal 50 exists is issued. Therefore, in a case where atoken is stored in the data storage area 62B (“No” in STEP S71), if theservice providing device 70 or 90 represented by the URL acquired inSTEP S51 is different from the service providing device 70 or 90corresponding to the area where the portable terminal 50 exists, thecontrol program 65 skips STEPS S72 to S76, and proceeds to STEP S79.

The location where the portable terminal 50 exists is specified by, forexample, location information which the portable terminal 50 hasreceived from a GPS (an abbreviation for global position system), orinformation of a communication device connected to the portable terminal50. As an example, in a case where the portable terminal 50 is connectedto a Wi-Fi network, the control program 65 may specify the location ofthe portable terminal 50 according to a global IP address set in abroadband router which is an example of the communication device. Asanother example, in a case where the portable terminal 50 is connectedto a mobile communication network, the control program 65 may specifythe location of the portable terminal 50 according to information on thelocation of the nearest base station.

The second acquiring process may be performed in a case where the firstacquiring process fails. Failure of the first acquiring process is, forexample, a case where it is impossible to acquire a first layoutdefinition data even though note request information has beentransmitted. In the case where it is impossible to acquire a firstlayout definition data is, for example, a case where first data has beenalready deleted from the service providing device 70, a case where atoken is invalid, or the like. As a result, in the case where it isimpossible to acquire a first layout definition data, it is possible toprevent a recording process from becoming totally impossible. Further,the second acquiring process may be performed in a case where adescription of a theme information is not included in the text datareferred to in STEP S51 (“No” in STEP S56).

The first and second embodiments can be combined. For example, as shownin FIG. 17 , in the note printing process according to the secondembodiment, in place of STEP S58, a printing control process (STEP S71)may be performed. The printing control process correspond to theprocesses of STEPS S18 to S25 of FIG. 4 , and thus will not be describedagain.

An example in which in a multi-function device 10 or the portableterminal 50 of the above described embodiment, the various programsstored in the program storage area 32A or 62A of the storage unit 32 or62 are executed by the CPU 31 or 61, whereby each process of the controlunit of the present disclosure is implemented has been described.However, the configuration of the control unit is limited thereto, andthe whole or a part of the control unit may be implemented by hardwaresuch as an IC (an abbreviation for Integrated Circuit) and the like.

Further, the aspect of the present disclosure can be implemented as themulti-function device 10 or the portable terminal 50, and can also beimplemented as a program making the multi-function device 10 or theportable terminal 50 perform the processes. Furthermore, thecorresponding program may be recorded in a non-transitory recordingmedium to be provided. The non-transitory recording medium may include astorage unit mounted on a server device which can be connected to themulti-function device 10 or the portable terminal 50 through thecommunication network 102, besides a CD-ROM, a DVD-ROM, or the like.Further, the program stored in the storage unit of the server device maybe distributed as information or a signal representing the correspondingprogram, through the communication network 102 such as the Internet.

What is claimed is:
 1. A terminal device comprising: a display unit; auser interface that receives touch operations on a display screen of thedisplay unit; a communication unit; and a processor configured toexecute: a note list displaying controlling process of controlling thedisplay unit to display a layout screen in which a list of a note imageicon indicating each of one or more notes is laid out; a touch operationreceiving process of receiving a touch operation to a location of thenote image icon by the user interface; a note information displayingcontrolling process of controlling the display unit to displayinformation of a note corresponding to a note image icon located wherethe touch operation is performed in the touch operation receivingprocess; a first print instruction receiving process of receiving aprint instruction of a note displayed in the note information displayingcontrolling process by the user interface; a first print instructionoutputting process of controlling the communication unit to output anote print instruction of printing the note based on a note associationvalue associated with the note, to a printer, according to a printinstruction received in the first print instruction receiving process; asecond print instruction receiving process of receiving a printinstruction of a first content included in a note displayed in the noteinformation displaying controlling process by the user interface; and asecond print instruction outputting process of controlling thecommunication unit to output a content print instruction of printing thecontent based on a content association value associated with the firstcontent, to the printer, according to a print instruction received inthe second print instruction receiving process.
 2. The terminal deviceaccording to claim 1, further comprising: a memory unit that stores thenote association value and the content association value.
 3. Theterminal device according to claim 1, wherein in the note informationdisplaying controlling process, the processor controls the display unitto display a content in a format being available for a print process anda content in a format being unavailable for a print process in differentdisplay modes.
 4. The terminal device according to claim 1, wherein theprocessor is configured to further execute: an acquisition process ofacquiring information of a note corresponding to a note image iconlocated where the touch operation is performed in the touch operationreceiving process from a service providing apparatus through thecommunication unit.
 5. The terminal device according to claim 1, whereinthe processor is configured to further execute: a third printinstruction receiving process of receiving a print instruction of asecond content included in a note displayed in the note informationdisplaying controlling process by the user interface; and a third printinstruction outputting process of controlling the communication unit tooutput a content print instruction of printing the content based on acontent association value associated with the second content, to theprinter, according to a print instruction received in the third printinstruction receiving process, and a format of the second content isdifferent from a format of the first content.
 6. The terminal deviceaccording to claim 1, wherein the note image icon in the layout screenincludes at least one of a thumbnail image and a format image.
 7. Theterminal device according to claim 3, wherein in the note informationdisplaying controlling process, the processor determines whether thefirst content is available for a print process, when it is determinedthat the first content is unavailable for the print process, theprocessor generates a non-processable image representing that the firstcontent is unavailable for the print process, and the processor controlsthe display unit to display the non-process sable image andidentification information for identifying the first content.
 8. Theterminal device according to claim 1, wherein the note association valueand the content association value include at least one of sheet typeoption value which defines a sheet type on which a print process isperformed and color option type which defines color characteristic ofthe print process.
 9. A non-transitory computer readable recordingmedium storing a computer readable instruction which can be executed bya computer which includes a display unit, a user interface for receivingtouch operations on a display screen of the display unit, and acommunication unit, the instruction when executed by the computer causesthe computer to execute: a note list displaying controlling process ofcontrolling the display unit to display a layout screen in which a listof a note image icon indicating each of one or more notes is laid out; atouch operation receiving process of receiving a touch operation to alocation of the note image icon by the user interface; a noteinformation displaying controlling process of controlling the displayunit to display information of a note corresponding to a note image iconlocated where the touch operation is performed in the touch operationreceiving process; a first print instruction receiving process ofreceiving a print instruction of a note displayed in the noteinformation displaying controlling process by the user interface; afirst print instruction outputting process of controlling thecommunication unit to output a note print instruction of printing thenote based on a note association value associated with the note, to aprinter, according to a print instruction received in the first printinstruction receiving process; a second print instruction receivingprocess of receiving a print instruction of a first content included ina note displayed in the note information displaying controlling processby the user interface; and a second print instruction outputting processof controlling the communication unit to output a content printinstruction of printing the content based on a content association valueassociated with the first content, to the printer, according to a printinstruction received in the second print instruction receiving process.10. An information processing method performed by a terminal deviceincluding a display unit, a user interface for receiving touchoperations on a display screen of the display unit, a communicationunit, and a processor, the method including the steps of executing: anote list displaying controlling process of controlling the display unitto display a layout screen in which a list of a note image iconindicating each of one or more notes is laid out; a touch operationreceiving process of receiving a touch operation to a location of thenote image icon by the user interface; a note information displayingcontrolling process of controlling the display unit to displayinformation of a note corresponding to a note image icon located wherethe touch operation is performed in the touch operation receivingprocess; a first print instruction receiving process of receiving aprint instruction of a note displayed in the note information displayingcontrolling process by the user interface; a first print instructionoutputting process of controlling the communication unit to output anote print instruction of printing the note based on a note associationvalue associated with the note, to a printer, according to a printinstruction received in the first print instruction receiving process; asecond print instruction receiving process of receiving a printinstruction of a first content included in a note displayed in the noteinformation displaying controlling process by the user interface; and asecond print instruction outputting process of controlling thecommunication unit to output a content print instruction of printing thecontent based on a content association value associated with the firstcontent, to the printer, according to a print instruction received inthe second print instruction receiving process.